Objectives

Introduction

In this tutorial, we will learn some R through creating volcano plots to visualise results from an RNA-seq experiment. We will demonstrate use of the tidyverse packages, such as ggplot2 for plotting and dplyr for manipulating tables, and go through the code step-by-step.

Analysis

First we will load the tidyverse library. This is already installed for you but if you need to install it on your own computer you can use install.packages(‘tidyverse’). The tidyverse package contains the dplyr and ggplot2 packages along with other packages we will use such as readr.

library(tidyverse)
library(ggrepel)

Next we load in our RNA-seq data. We will use the published data for 12 samples from mouse luminal and basal cells (ref). The file we will use is tab-separated, so we will use the read_tsv() function from the tidyverse readr package to read it in. We will store the contents of our file in a variable called de_results.

de_results <- read_tsv("https://zenodo.org/record/2596382/files/limma-voom_luminalpregnant-luminallactate")
Parsed with column specification:
cols(
  ENTREZID = col_double(),
  SYMBOL = col_character(),
  GENENAME = col_character(),
  logFC = col_double(),
  AveExpr = col_double(),
  t = col_double(),
  P.Value = col_double(),
  adj.P.Val = col_double(),
  B = col_double()
)

Next we can check the number of rows and columns in our file using R’s dim()function.

dim(de_results)
[1] 15804     9

There are 15,804 rows and 8 columns.

We will use the R head() function to look at the first few rows, it shows 6 by default.

head(de_results)
# A tibble: 6 x 9
  ENTREZID SYMBOL  GENENAME                                                      logFC AveExpr     t  P.Value adj.P.Val     B
     <dbl> <chr>   <chr>                                                         <dbl>   <dbl> <dbl>    <dbl>     <dbl> <dbl>
1    12992 Csn1s2b casein alpha s2-like B                                        -8.60    3.56 -47.0 2.73e-15  4.32e-11  24.3
2    13358 Slc25a1 solute carrier family 25 (mitochondrial carrier, citrate tra… -4.12    5.78 -34.8 5.06e-14  4.00e-10  22.5
3    20531 Slc34a2 solute carrier family 34 (sodium phosphate), member 2         -4.18    4.28 -31.4 1.83e-13  9.62e-10  21.3
4    11941 Atp2b2  ATPase, Ca++ transporting, plasma membrane 2                  -7.39    1.28 -30.2 3.04e-13  1.20e- 9  19.6
5   100705 Acacb   acetyl-Coenzyme A carboxylase beta                            -4.31    4.44 -29.1 4.74e-13  1.29e- 9  20.3
6   230810 Slc30a2 solute carrier family 30 (zinc transporter), member 2         -3.20    2.70 -29.1 4.90e-13  1.29e- 9  20.3

Or look at all the data with View().

View(de_results)

To make our volcano plot we make a scatterplot with logFC on the x axis and -log10(P.Value) on the y axis.

ggplot(de_results, aes(logFC, -log10(P.Value))) +
    geom_point()

Let’s colour points that are significant at adj.P.Value < 0.05. To do this we use mutate() to add a column called “sig” and we use ifelse() to say if our adj.P.Val is below 0.05 we add “Sig” to the sig column, othewise add “Not sig”.

de_results <- de_results %>%  
  mutate(sig=ifelse(adj.P.Val < 0.05, "Sig", "Not sig"))

Let’s take a look at the de_results now. We should see we have a new column at the end called “sig”.

head(de_results)
# A tibble: 6 x 10
  ENTREZID SYMBOL  GENENAME                                                logFC AveExpr     t  P.Value adj.P.Val     B sig  
     <dbl> <chr>   <chr>                                                   <dbl>   <dbl> <dbl>    <dbl>     <dbl> <dbl> <chr>
1    12992 Csn1s2b casein alpha s2-like B                                  -8.60    3.56 -47.0 2.73e-15  4.32e-11  24.3 Sig  
2    13358 Slc25a1 solute carrier family 25 (mitochondrial carrier, citra… -4.12    5.78 -34.8 5.06e-14  4.00e-10  22.5 Sig  
3    20531 Slc34a2 solute carrier family 34 (sodium phosphate), member 2   -4.18    4.28 -31.4 1.83e-13  9.62e-10  21.3 Sig  
4    11941 Atp2b2  ATPase, Ca++ transporting, plasma membrane 2            -7.39    1.28 -30.2 3.04e-13  1.20e- 9  19.6 Sig  
5   100705 Acacb   acetyl-Coenzyme A carboxylase beta                      -4.31    4.44 -29.1 4.74e-13  1.29e- 9  20.3 Sig  
6   230810 Slc30a2 solute carrier family 30 (zinc transporter), member 2   -3.20    2.70 -29.1 4.90e-13  1.29e- 9  20.3 Sig  

Have a closer look with View() and sort on the sig column to check you see both Sig and Not sig entries.

Now we can colour the significant genes using our sig column by adding col=sig.

ggplot(de_results, aes(logFC, -log10(P.Value), col=sig)) +
    geom_point()

We might want to change the colours, for example, to make the non-significant grey and the significant red. To do that we add + scale_colour_manual.

ggplot(de_results, aes(logFC, -log10(P.Value), col=sig)) +
    geom_point() +
  scale_colour_manual(values=c("red", "grey"))

Hmm this is the wrong way around, our significant points are grey. We’d be better to specify which category in our sig column we want to map to each colour, so let’s do that.

ggplot(de_results, aes(logFC, -log10(P.Value), col=sig)) +
    geom_point() +
  scale_colour_manual(values=c("Sig"="red", "Not sig"="grey"))

We could colour our signficant genes that are downregulated and the genes thare are upregulated using separate colours, by changing our sig column. Let’s colour significant genes > logFC of 1 red and < logFC -1 blue. To do this we change our sig column by adding another ifelse. If our genes are < 0.05 and also have a logFC > 1 we label then “Up”, if they are < 0.05 and have a logFC < -1 we label them “Down”, otherwise we label them “Not sig”.

de_results <- de_results %>%  
  mutate(sig=ifelse((adj.P.Val < 0.05 & logFC > 1), "Up", ifelse((adj.P.Val < 0.05 & logFC < -1), "Down", "Not sig")))

Let’s take a look at the output.

head(de_results)
# A tibble: 6 x 10
  ENTREZID SYMBOL  GENENAME                                                logFC AveExpr     t  P.Value adj.P.Val     B sig  
     <dbl> <chr>   <chr>                                                   <dbl>   <dbl> <dbl>    <dbl>     <dbl> <dbl> <chr>
1    12992 Csn1s2b casein alpha s2-like B                                  -8.60    3.56 -47.0 2.73e-15  4.32e-11  24.3 Down 
2    13358 Slc25a1 solute carrier family 25 (mitochondrial carrier, citra… -4.12    5.78 -34.8 5.06e-14  4.00e-10  22.5 Down 
3    20531 Slc34a2 solute carrier family 34 (sodium phosphate), member 2   -4.18    4.28 -31.4 1.83e-13  9.62e-10  21.3 Down 
4    11941 Atp2b2  ATPase, Ca++ transporting, plasma membrane 2            -7.39    1.28 -30.2 3.04e-13  1.20e- 9  19.6 Down 
5   100705 Acacb   acetyl-Coenzyme A carboxylase beta                      -4.31    4.44 -29.1 4.74e-13  1.29e- 9  20.3 Down 
6   230810 Slc30a2 solute carrier family 30 (zinc transporter), member 2   -3.20    2.70 -29.1 4.90e-13  1.29e- 9  20.3 Down 

We can colour the volcano plot points, up - red, not signif - grey, and down - blue.

ggplot(de_results, aes(logFC, -log10(P.Value), col=sig)) +
    geom_point() +
  scale_colour_manual(values=c("Up"="red", "Not sig"="grey", "Down"="blue"))

We could label the top significant genes with the gene names so we can see what they are. We have gene symbols in our de_results so we could use those.

Let’s label the top 10 most significant genes. To do this we first identity the top 10 genes by P.Value.

# create labels for top values
top_10 <- de_results %>%
  top_n(10, -P.Value)

Next we add a column to say whether to label the point or not.

de_results <- mutate(de_results, labels=ifelse(SYMBOL %in% top_10$SYMBOL, SYMBOL, ""))

Take a look.

head(de_results)
# A tibble: 6 x 11
  ENTREZID SYMBOL  GENENAME                                         logFC AveExpr     t  P.Value adj.P.Val     B sig   labels
     <dbl> <chr>   <chr>                                            <dbl>   <dbl> <dbl>    <dbl>     <dbl> <dbl> <chr> <chr> 
1    12992 Csn1s2b casein alpha s2-like B                           -8.60    3.56 -47.0 2.73e-15  4.32e-11  24.3 Down  Csn1s…
2    13358 Slc25a1 solute carrier family 25 (mitochondrial carrier… -4.12    5.78 -34.8 5.06e-14  4.00e-10  22.5 Down  Slc25…
3    20531 Slc34a2 solute carrier family 34 (sodium phosphate), me… -4.18    4.28 -31.4 1.83e-13  9.62e-10  21.3 Down  Slc34…
4    11941 Atp2b2  ATPase, Ca++ transporting, plasma membrane 2     -7.39    1.28 -30.2 3.04e-13  1.20e- 9  19.6 Down  Atp2b2
5   100705 Acacb   acetyl-Coenzyme A carboxylase beta               -4.31    4.44 -29.1 4.74e-13  1.29e- 9  20.3 Down  Acacb 
6   230810 Slc30a2 solute carrier family 30 (zinc transporter), me… -3.20    2.70 -29.1 4.90e-13  1.29e- 9  20.3 Down  Slc30…

Next we add + geom_text() and add the labels into that.

ggplot(de_results, aes(logFC, -log10(P.Value), col=sig)) +
  geom_point() +
  scale_colour_manual(values=c("Up"="red", "Not sig"="grey", "Down"="blue")) +
  geom_text(aes(label=labels))

That doesn’t look great as the labels are overlapping but we fix that. We can replace + geom_text() with + geom_text_repel() from the package ggrepel.

ggplot(de_results, aes(logFC, -log10(P.Value), col=sig)) +
  geom_point() +
  scale_colour_manual(values=c("Up"="red", "Not sig"="grey", "Down"="blue")) +
  geom_text_repel(aes(label=labels))

The legend now has a legend for the labels overlapping but we can remove that by adding show.legend=FALSE.

ggplot(de_results, aes(logFC, -log10(P.Value), col=sig)) +
  geom_point() +
  scale_colour_manual(values=c("Up"="red", "Not sig"="grey", "Down"="blue")) +
  geom_text_repel(aes(label=labels), show.legend = FALSE)

Modifications to plot

If we want, we can just colour the points red and blue, and leave the labels black, by adding the col=sig into the geom_point() aes() instead of in the ggplot() aes().

ggplot(de_results, aes(logFC, -log10(P.Value))) +
  geom_point(aes(col=sig)) +
  scale_colour_manual(values=c("Up"="red", "Not sig"="grey", "Down"="blue")) +
  geom_text_repel(aes(label=labels), show.legend = FALSE)

We can also make the plot look nicer, for example, adding a title and changing the background. First let’s store our inital plot in a variable called p, to make it easier to see what we’re changing.

p <- ggplot(de_results, aes(logFC, -log10(P.Value), col=sig)) +
  geom_point() +
  scale_colour_manual(values=c("Up"="red", "Not sig"="grey", "Down"="blue")) +
  geom_text_repel(aes(label=labels), show.legend = FALSE)

We can add a title.

p <- p + labs(title="Luminal pregnant vs lactating")
p

We can centre the title if we prefer.

p <- p + theme(plot.title = element_text(hjust = 0.5))
p

We can remove the grey background and grid lines.

p <- p + theme(panel.background = element_blank(), 
               panel.grid.major = element_blank(), 
               panel.grid.minor = element_blank())
p

We can adjust the x axis so it’s the same limit on the right and left.

p <- p + scale_x_continuous(limits=c(-10, 10))
p

Exercise

Make a volcano plot for the basal cells using the file “https://zenodo.org/record/2596382/files/limma-voom_basalpregnant-basallactate”. You can choose to colour and modify it whatever way you like.

Further Reading

An Introduction to ggplot by Babraham Bioinformatics
Data Manipulation and Visualisation by University of Cambridge

LS0tCnRpdGxlOiAiSW50cm9kdWN0aW9uIHRvIFIiCmF1dGhvcjogIk1hcmlhIERveWxlIgpkYXRlOiAiYHIgZm9ybWF0KFN5cy50aW1lKCksICclZCAlQiAlWScpYCIKb3V0cHV0OgogIGh0bWxfbm90ZWJvb2s6IGRlZmF1bHQKICBodG1sX2RvY3VtZW50OiBkZWZhdWx0CnN1YnRpdGxlOiB2aXN1YWxpc2luZyBSTkEtc2VxIGRhdGEgd2l0aCB0aGUgdGlkeXZlcnNlICh2b2xjYW5vIHBsb3QpCi0tLQoKIyMgT2JqZWN0aXZlcwoKKiBEZW1vbnN0cmF0ZSBob3cgUiBjYW4gYmUgdXNlZCB0byB2aXN1YWxpc2UgUk5BLVNlcSBkYXRhCiogRGVtb25zdHJhdGUgYmFzaWMgUiBzeW50YXggKGA8LWAsIGAlaW4lYCxgIGMoKWAsIGBkaW0oKWAsIGBoZWFkKClgLCBgVmlldygpYCkKKiBVdGlsaXplIGRwbHlyIGZvciBlZmZpY2llbnQgbWFuaXB1bGF0aW9uIG9mIHRhYmxlcyAoYGZpbHRlcmAgYW5kIGBtdXRhdGVgIGFuZCB0aGUgaW5jcmVkaWJseSBwb3dlcmZ1bCBwaXBlIGAlPiVgKS4KKiBVdGlsaXplIGdncGxvdDIgYGdlb21fcG9pbnQoKWAgYW5kIGBnZ3Bsb3RfdGV4dF9yZXBlbGAoKQoKIyMgSW50cm9kdWN0aW9uCgpJbiB0aGlzIHR1dG9yaWFsLCB3ZSB3aWxsIGxlYXJuIHNvbWUgUiB0aHJvdWdoIGNyZWF0aW5nIHZvbGNhbm8gcGxvdHMgdG8gdmlzdWFsaXNlIHJlc3VsdHMgZnJvbSBhbiBSTkEtc2VxIGV4cGVyaW1lbnQuIFdlIHdpbGwgZGVtb25zdHJhdGUgdXNlIG9mIHRoZSAqKnRpZHl2ZXJzZSoqIHBhY2thZ2VzLCBzdWNoIGFzICoqZ2dwbG90MioqIGZvciBwbG90dGluZyBhbmQgKipkcGx5cioqIGZvciBtYW5pcHVsYXRpbmcgdGFibGVzLCBhbmQgZ28gdGhyb3VnaCB0aGUgY29kZSBzdGVwLWJ5LXN0ZXAuCgojIyBBbmFseXNpcwoKRmlyc3Qgd2Ugd2lsbCBsb2FkIHRoZSB0aWR5dmVyc2UgbGlicmFyeS4gVGhpcyBpcyBhbHJlYWR5IGluc3RhbGxlZCBmb3IgeW91IGJ1dCBpZiB5b3UgbmVlZCB0byBpbnN0YWxsIGl0IG9uIHlvdXIgb3duIGNvbXB1dGVyIHlvdSBjYW4gdXNlIGluc3RhbGwucGFja2FnZXMoJ3RpZHl2ZXJzZScpLiBUaGUgdGlkeXZlcnNlIHBhY2thZ2UgY29udGFpbnMgdGhlIGRwbHlyIGFuZCBnZ3Bsb3QyIHBhY2thZ2VzIGFsb25nIHdpdGggb3RoZXIgcGFja2FnZXMgd2Ugd2lsbCB1c2Ugc3VjaCBhcyAqKnJlYWRyKiouIAoKYGBge3J9CmxpYnJhcnkodGlkeXZlcnNlKQpsaWJyYXJ5KGdncmVwZWwpCmBgYAoKTmV4dCB3ZSBsb2FkIGluIG91ciBSTkEtc2VxIGRhdGEuIFdlIHdpbGwgdXNlIHRoZSBwdWJsaXNoZWQgZGF0YSBmb3IgMTIgc2FtcGxlcyBmcm9tIG1vdXNlIGx1bWluYWwgYW5kIGJhc2FsIGNlbGxzIChyZWYpLiBUaGUgZmlsZSB3ZSB3aWxsIHVzZSBpcyB0YWItc2VwYXJhdGVkLCBzbyB3ZSB3aWxsIHVzZSB0aGUgYHJlYWRfdHN2KClgIGZ1bmN0aW9uIGZyb20gdGhlIHRpZHl2ZXJzZSByZWFkciBwYWNrYWdlIHRvIHJlYWQgaXQgaW4uIFdlIHdpbGwgc3RvcmUgdGhlIGNvbnRlbnRzIG9mIG91ciBmaWxlIGluIGEgdmFyaWFibGUgY2FsbGVkIGBkZV9yZXN1bHRzYC4KCmBgYHtyfQpkZV9yZXN1bHRzIDwtIHJlYWRfdHN2KCJodHRwczovL3plbm9kby5vcmcvcmVjb3JkLzI1OTYzODIvZmlsZXMvbGltbWEtdm9vbV9sdW1pbmFscHJlZ25hbnQtbHVtaW5hbGxhY3RhdGUiKQpgYGAKCk5leHQgd2UgY2FuIGNoZWNrIHRoZSBudW1iZXIgb2Ygcm93cyBhbmQgY29sdW1ucyBpbiBvdXIgZmlsZSB1c2luZyBSJ3MgYGRpbSgpYGZ1bmN0aW9uLgoKYGBge3J9CmRpbShkZV9yZXN1bHRzKQpgYGAKClRoZXJlIGFyZSAxNSw4MDQgcm93cyBhbmQgOCBjb2x1bW5zLiAKCldlIHdpbGwgdXNlIHRoZSBSIGBoZWFkKClgIGZ1bmN0aW9uICB0byBsb29rIGF0IHRoZSBmaXJzdCBmZXcgcm93cywgaXQgc2hvd3MgNiBieSBkZWZhdWx0LgpgYGB7cn0KaGVhZChkZV9yZXN1bHRzKQpgYGAKCk9yIGxvb2sgYXQgYWxsIHRoZSBkYXRhIHdpdGggYFZpZXcoKWAuCgpgYGB7ciBldmFsPUZBTFNFfQpWaWV3KGRlX3Jlc3VsdHMpCmBgYAoKVG8gbWFrZSBvdXIgdm9sY2FubyBwbG90IHdlIG1ha2UgYSBzY2F0dGVycGxvdCB3aXRoIGxvZ0ZDIG9uIHRoZSB4IGF4aXMgYW5kIC1sb2cxMChQLlZhbHVlKSBvbiB0aGUgeSBheGlzLgoKYGBge3J9CmdncGxvdChkZV9yZXN1bHRzLCBhZXMobG9nRkMsIC1sb2cxMChQLlZhbHVlKSkpICsKICAgIGdlb21fcG9pbnQoKQpgYGAKCkxldCdzIGNvbG91ciBwb2ludHMgdGhhdCBhcmUgc2lnbmlmaWNhbnQgYXQgYWRqLlAuVmFsdWUgPCAwLjA1LiBUbyBkbyB0aGlzIHdlIHVzZSBgbXV0YXRlKClgIHRvIGFkZCBhIGNvbHVtbiBjYWxsZWQgInNpZyIgYW5kIHdlIHVzZSBgaWZlbHNlKClgIHRvIHNheSBpZiBvdXIgYWRqLlAuVmFsIGlzIGJlbG93IDAuMDUgd2UgYWRkICJTaWciIHRvIHRoZSBzaWcgY29sdW1uLCBvdGhld2lzZSBhZGQgIk5vdCBzaWciLiAKCmBgYHtyfQpkZV9yZXN1bHRzIDwtIGRlX3Jlc3VsdHMgJT4lICAKICBtdXRhdGUoc2lnPWlmZWxzZShhZGouUC5WYWwgPCAwLjA1LCAiU2lnIiwgIk5vdCBzaWciKSkKYGBgCgpMZXQncyB0YWtlIGEgbG9vayBhdCB0aGUgZGVfcmVzdWx0cyBub3cuIFdlIHNob3VsZCBzZWUgd2UgaGF2ZSBhIG5ldyBjb2x1bW4gYXQgdGhlIGVuZCBjYWxsZWQgInNpZyIuCgpgYGB7cn0KaGVhZChkZV9yZXN1bHRzKQpgYGAKCkhhdmUgYSBjbG9zZXIgbG9vayB3aXRoIFZpZXcoKSBhbmQgc29ydCBvbiB0aGUgc2lnIGNvbHVtbiB0byBjaGVjayB5b3Ugc2VlIGJvdGggU2lnIGFuZCBOb3Qgc2lnIGVudHJpZXMuCgpOb3cgd2UgY2FuIGNvbG91ciB0aGUgc2lnbmlmaWNhbnQgZ2VuZXMgdXNpbmcgb3VyIHNpZyBjb2x1bW4gYnkgYWRkaW5nIGNvbD1zaWcuCgpgYGB7cn0KZ2dwbG90KGRlX3Jlc3VsdHMsIGFlcyhsb2dGQywgLWxvZzEwKFAuVmFsdWUpLCBjb2w9c2lnKSkgKwogICAgZ2VvbV9wb2ludCgpCmBgYAoKV2UgbWlnaHQgd2FudCB0byBjaGFuZ2UgdGhlIGNvbG91cnMsIGZvciBleGFtcGxlLCB0byBtYWtlIHRoZSBub24tc2lnbmlmaWNhbnQgZ3JleSBhbmQgdGhlIHNpZ25pZmljYW50IHJlZC4gVG8gZG8gdGhhdCB3ZSBhZGQgYCsgc2NhbGVfY29sb3VyX21hbnVhbGAuCgpgYGB7cn0KZ2dwbG90KGRlX3Jlc3VsdHMsIGFlcyhsb2dGQywgLWxvZzEwKFAuVmFsdWUpLCBjb2w9c2lnKSkgKwogICAgZ2VvbV9wb2ludCgpICsKICBzY2FsZV9jb2xvdXJfbWFudWFsKHZhbHVlcz1jKCJyZWQiLCAiZ3JleSIpKQpgYGAKCkhtbSB0aGlzIGlzIHRoZSB3cm9uZyB3YXkgYXJvdW5kLCBvdXIgc2lnbmlmaWNhbnQgcG9pbnRzIGFyZSBncmV5LiBXZSdkIGJlIGJldHRlciB0byBzcGVjaWZ5IHdoaWNoIGNhdGVnb3J5IGluIG91ciBzaWcgY29sdW1uIHdlIHdhbnQgdG8gbWFwIHRvIGVhY2ggY29sb3VyLCBzbyBsZXQncyBkbyB0aGF0LgoKYGBge3J9CmdncGxvdChkZV9yZXN1bHRzLCBhZXMobG9nRkMsIC1sb2cxMChQLlZhbHVlKSwgY29sPXNpZykpICsKICAgIGdlb21fcG9pbnQoKSArCiAgc2NhbGVfY29sb3VyX21hbnVhbCh2YWx1ZXM9YygiU2lnIj0icmVkIiwgIk5vdCBzaWciPSJncmV5IikpCmBgYAoKV2UgY291bGQgY29sb3VyIG91ciBzaWduZmljYW50IGdlbmVzIHRoYXQgYXJlIGRvd25yZWd1bGF0ZWQgYW5kIHRoZSBnZW5lcyB0aGFyZSBhcmUgdXByZWd1bGF0ZWQgdXNpbmcgc2VwYXJhdGUgY29sb3VycywgYnkgY2hhbmdpbmcgb3VyIHNpZyBjb2x1bW4uIApMZXQncyBjb2xvdXIgc2lnbmlmaWNhbnQgZ2VuZXMgPiBsb2dGQyBvZiAxIHJlZCBhbmQgPCBsb2dGQyAtMSBibHVlLiBUbyBkbyB0aGlzIHdlIGNoYW5nZSBvdXIgc2lnIGNvbHVtbiBieSBhZGRpbmcgYW5vdGhlciBpZmVsc2UuIElmIG91ciBnZW5lcyBhcmUgPCAwLjA1IGFuZCBhbHNvIGhhdmUgYSBsb2dGQyA+IDEgd2UgbGFiZWwgdGhlbiAiVXAiLCBpZiB0aGV5IGFyZSA8IDAuMDUgYW5kIGhhdmUgYSBsb2dGQyA8IC0xIHdlIGxhYmVsIHRoZW0gIkRvd24iLCBvdGhlcndpc2Ugd2UgbGFiZWwgdGhlbSAiTm90IHNpZyIuCgpgYGB7cn0KZGVfcmVzdWx0cyA8LSBkZV9yZXN1bHRzICU+JSAgCiAgbXV0YXRlKHNpZz1pZmVsc2UoKGFkai5QLlZhbCA8IDAuMDUgJiBsb2dGQyA+IDEpLCAiVXAiLCBpZmVsc2UoKGFkai5QLlZhbCA8IDAuMDUgJiBsb2dGQyA8IC0xKSwgIkRvd24iLCAiTm90IHNpZyIpKSkKYGBgCgpMZXQncyB0YWtlIGEgbG9vayBhdCB0aGUgb3V0cHV0LgoKYGBge3J9CmhlYWQoZGVfcmVzdWx0cykKYGBgCgpXZSBjYW4gY29sb3VyIHRoZSB2b2xjYW5vIHBsb3QgcG9pbnRzLCB1cCAtIHJlZCwgbm90IHNpZ25pZiAtIGdyZXksIGFuZCBkb3duIC0gYmx1ZS4KCmBgYHtyfQpnZ3Bsb3QoZGVfcmVzdWx0cywgYWVzKGxvZ0ZDLCAtbG9nMTAoUC5WYWx1ZSksIGNvbD1zaWcpKSArCiAgICBnZW9tX3BvaW50KCkgKwogIHNjYWxlX2NvbG91cl9tYW51YWwodmFsdWVzPWMoIlVwIj0icmVkIiwgIk5vdCBzaWciPSJncmV5IiwgIkRvd24iPSJibHVlIikpCmBgYAoKV2UgY291bGQgbGFiZWwgdGhlIHRvcCBzaWduaWZpY2FudCBnZW5lcyB3aXRoIHRoZSBnZW5lIG5hbWVzIHNvIHdlIGNhbiBzZWUgd2hhdCB0aGV5IGFyZS4gV2UgaGF2ZSBnZW5lIHN5bWJvbHMgaW4gb3VyIGRlX3Jlc3VsdHMgc28gd2UgY291bGQgdXNlIHRob3NlLiAKCkxldCdzIGxhYmVsIHRoZSB0b3AgMTAgbW9zdCBzaWduaWZpY2FudCBnZW5lcy4gVG8gZG8gdGhpcyB3ZSBmaXJzdCBpZGVudGl0eSB0aGUgdG9wIDEwIGdlbmVzIGJ5IFAuVmFsdWUuCgpgYGB7cn0KIyBjcmVhdGUgbGFiZWxzIGZvciB0b3AgdmFsdWVzCnRvcF8xMCA8LSBkZV9yZXN1bHRzICU+JQogIHRvcF9uKDEwLCAtUC5WYWx1ZSkKYGBgCgpOZXh0IHdlIGFkZCBhIGNvbHVtbiB0byBzYXkgd2hldGhlciB0byBsYWJlbCB0aGUgcG9pbnQgb3Igbm90LgoKYGBge3J9CmRlX3Jlc3VsdHMgPC0gbXV0YXRlKGRlX3Jlc3VsdHMsIGxhYmVscz1pZmVsc2UoU1lNQk9MICVpbiUgdG9wXzEwJFNZTUJPTCwgU1lNQk9MLCAiIikpCmBgYAoKVGFrZSBhIGxvb2suCgpgYGB7cn0KaGVhZChkZV9yZXN1bHRzKQpgYGAKCk5leHQgd2UgYWRkIGArIGdlb21fdGV4dCgpYCBhbmQgYWRkIHRoZSBsYWJlbHMgaW50byB0aGF0LgoKYGBge3J9CmdncGxvdChkZV9yZXN1bHRzLCBhZXMobG9nRkMsIC1sb2cxMChQLlZhbHVlKSwgY29sPXNpZykpICsKICBnZW9tX3BvaW50KCkgKwogIHNjYWxlX2NvbG91cl9tYW51YWwodmFsdWVzPWMoIlVwIj0icmVkIiwgIk5vdCBzaWciPSJncmV5IiwgIkRvd24iPSJibHVlIikpICsKICBnZW9tX3RleHQoYWVzKGxhYmVsPWxhYmVscykpCmBgYAoKVGhhdCBkb2Vzbid0IGxvb2sgZ3JlYXQgYXMgdGhlIGxhYmVscyBhcmUgb3ZlcmxhcHBpbmcgYnV0IHdlIGZpeCB0aGF0LiBXZSBjYW4gcmVwbGFjZSBgKyBnZW9tX3RleHQoKWAgd2l0aCBgKyBnZW9tX3RleHRfcmVwZWwoKWAgZnJvbSB0aGUgcGFja2FnZSBnZ3JlcGVsLgoKYGBge3J9CmdncGxvdChkZV9yZXN1bHRzLCBhZXMobG9nRkMsIC1sb2cxMChQLlZhbHVlKSwgY29sPXNpZykpICsKICBnZW9tX3BvaW50KCkgKwogIHNjYWxlX2NvbG91cl9tYW51YWwodmFsdWVzPWMoIlVwIj0icmVkIiwgIk5vdCBzaWciPSJncmV5IiwgIkRvd24iPSJibHVlIikpICsKICBnZW9tX3RleHRfcmVwZWwoYWVzKGxhYmVsPWxhYmVscykpCmBgYAoKVGhlIGxlZ2VuZCBub3cgaGFzIGEgbGVnZW5kIGZvciB0aGUgbGFiZWxzIG92ZXJsYXBwaW5nIGJ1dCB3ZSBjYW4gcmVtb3ZlIHRoYXQgYnkgYWRkaW5nIGBzaG93LmxlZ2VuZD1GQUxTRWAuCgpgYGB7cn0KZ2dwbG90KGRlX3Jlc3VsdHMsIGFlcyhsb2dGQywgLWxvZzEwKFAuVmFsdWUpLCBjb2w9c2lnKSkgKwogIGdlb21fcG9pbnQoKSArCiAgc2NhbGVfY29sb3VyX21hbnVhbCh2YWx1ZXM9YygiVXAiPSJyZWQiLCAiTm90IHNpZyI9ImdyZXkiLCAiRG93biI9ImJsdWUiKSkgKwogIGdlb21fdGV4dF9yZXBlbChhZXMobGFiZWw9bGFiZWxzKSwgc2hvdy5sZWdlbmQgPSBGQUxTRSkKYGBgCgojIyBNb2RpZmljYXRpb25zIHRvIHBsb3QKCklmIHdlIHdhbnQsIHdlIGNhbiBqdXN0IGNvbG91ciB0aGUgcG9pbnRzIHJlZCBhbmQgYmx1ZSwgYW5kIGxlYXZlIHRoZSBsYWJlbHMgYmxhY2ssIGJ5IGFkZGluZyB0aGUgYGNvbD1zaWdgIGludG8gdGhlIGBnZW9tX3BvaW50KClgIGBhZXMoKWAgaW5zdGVhZCBvZiBpbiB0aGUgYGdncGxvdCgpYCBgYWVzKClgLgoKYGBge3J9CmdncGxvdChkZV9yZXN1bHRzLCBhZXMobG9nRkMsIC1sb2cxMChQLlZhbHVlKSkpICsKICBnZW9tX3BvaW50KGFlcyhjb2w9c2lnKSkgKwogIHNjYWxlX2NvbG91cl9tYW51YWwodmFsdWVzPWMoIlVwIj0icmVkIiwgIk5vdCBzaWciPSJncmV5IiwgIkRvd24iPSJibHVlIikpICsKICBnZW9tX3RleHRfcmVwZWwoYWVzKGxhYmVsPWxhYmVscyksIHNob3cubGVnZW5kID0gRkFMU0UpCmBgYAoKV2UgY2FuIGFsc28gbWFrZSB0aGUgcGxvdCBsb29rIG5pY2VyLCBmb3IgZXhhbXBsZSwgYWRkaW5nIGEgdGl0bGUgYW5kIGNoYW5naW5nIHRoZSBiYWNrZ3JvdW5kLiBGaXJzdCBsZXQncyBzdG9yZSBvdXIgaW5pdGFsIHBsb3QgaW4gYSB2YXJpYWJsZSBjYWxsZWQgYHBgLCB0byBtYWtlIGl0IGVhc2llciB0byBzZWUgd2hhdCB3ZSdyZSBjaGFuZ2luZy4KCmBgYHtyfQpwIDwtIGdncGxvdChkZV9yZXN1bHRzLCBhZXMobG9nRkMsIC1sb2cxMChQLlZhbHVlKSwgY29sPXNpZykpICsKICBnZW9tX3BvaW50KCkgKwogIHNjYWxlX2NvbG91cl9tYW51YWwodmFsdWVzPWMoIlVwIj0icmVkIiwgIk5vdCBzaWciPSJncmV5IiwgIkRvd24iPSJibHVlIikpICsKICBnZW9tX3RleHRfcmVwZWwoYWVzKGxhYmVsPWxhYmVscyksIHNob3cubGVnZW5kID0gRkFMU0UpCmBgYAoKV2UgY2FuIGFkZCBhIHRpdGxlLgoKYGBge3J9CnAgPC0gcCArIGxhYnModGl0bGU9Ikx1bWluYWwgcHJlZ25hbnQgdnMgbGFjdGF0aW5nIikKcApgYGAKCldlIGNhbiBjZW50cmUgdGhlIHRpdGxlIGlmIHdlIHByZWZlci4KCmBgYHtyfQpwIDwtIHAgKyB0aGVtZShwbG90LnRpdGxlID0gZWxlbWVudF90ZXh0KGhqdXN0ID0gMC41KSkKcApgYGAKCldlIGNhbiByZW1vdmUgdGhlIGdyZXkgYmFja2dyb3VuZCBhbmQgZ3JpZCBsaW5lcy4KCmBgYHtyfQpwIDwtIHAgKyB0aGVtZShwYW5lbC5iYWNrZ3JvdW5kID0gZWxlbWVudF9ibGFuaygpLCAKICAgICAgICAgICAgICAgcGFuZWwuZ3JpZC5tYWpvciA9IGVsZW1lbnRfYmxhbmsoKSwgCiAgICAgICAgICAgICAgIHBhbmVsLmdyaWQubWlub3IgPSBlbGVtZW50X2JsYW5rKCkpCnAKYGBgCgpXZSBjYW4gYWRqdXN0IHRoZSB4IGF4aXMgc28gaXQncyB0aGUgc2FtZSBsaW1pdCBvbiB0aGUgcmlnaHQgYW5kIGxlZnQuCgpgYGB7cn0KcCA8LSBwICsgc2NhbGVfeF9jb250aW51b3VzKGxpbWl0cz1jKC0xMCwgMTApKQpwCmBgYAoKCiMjIyBFeGVyY2lzZQoKTWFrZSBhIHZvbGNhbm8gcGxvdCBmb3IgdGhlIGJhc2FsIGNlbGxzIHVzaW5nIHRoZSBmaWxlICJodHRwczovL3plbm9kby5vcmcvcmVjb3JkLzI1OTYzODIvZmlsZXMvbGltbWEtdm9vbV9iYXNhbHByZWduYW50LWJhc2FsbGFjdGF0ZSIuIFlvdSBjYW4gY2hvb3NlIHRvIGNvbG91ciBhbmQgbW9kaWZ5IGl0IHdoYXRldmVyIHdheSB5b3UgbGlrZS4KCiMjIEZ1cnRoZXIgUmVhZGluZwpbQW4gSW50cm9kdWN0aW9uIHRvIGdncGxvdF0oaHR0cHM6Ly93d3cuYmlvaW5mb3JtYXRpY3MuYmFicmFoYW0uYWMudWsvdHJhaW5pbmcuaHRtbCNnZ3Bsb3QpIGJ5IEJhYnJhaGFtIEJpb2luZm9ybWF0aWNzICAKW0RhdGEgTWFuaXB1bGF0aW9uIGFuZCBWaXN1YWxpc2F0aW9uXShodHRwOi8vYmlvaW5mb3JtYXRpY3MtY29yZS1zaGFyZWQtdHJhaW5pbmcuZ2l0aHViLmlvL3ItaW50ZXJtZWRpYXRlLykgYnkgVW5pdmVyc2l0eSBvZiBDYW1icmlkZ2Ug